Communication lays the foundation for human cooperation. It is also crucial for multi-agent cooperation. However, existing work focuses on broadcast communication, which is not only impractical but also leads to information redundancy that could even impair the learning process. To tackle these difficulties, we propose Individually Inferred Communication (I2C), a simple yet effective model to enable agents to learn a prior for agent-agent communication. The prior knowledge is learned via causal inference and realized by a feed-forward neural network that maps the agent's local observation to a belief about who to communicate with. The influence of one agent on another is inferred via the joint action-value function in multi-agent reinforcement learning and quantified to label the necessity of agent-agent communication. Furthermore, the agent policy is regularized to better exploit communicated messages. Empirically, we show that I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios, comparing to existing methods.

Summary and Contributions: This paper introduces I2C, a multi-agent communication architecture for cooperative tasks wherein each agent decides who to receive messages from. This is unlike prior work in multi-agent communication that has primarily focused on broadcast-style communication -- one/all agents sending messages to all other agents. The motivation is to reduce redundant communication (which might ease learning) and make the overall setup more practically realizable. I2C consists of a "prior network", which takes as input agent i's observation and predicts a probability distribution of which other agents to receive messages from. This prior network is trained with supervised learning to minimize the KL divergence between probability of the agent's action given the actions of agents other than i and probability of the agent's action given actions of agents other than i and j; the idea being that the prior network should enable communication only from those agents who might have a strong influence on agent i's action.

All reviewers support acceptance of this paper, and I would also like to recommend acceptance. All reviewers point that this is an interesting a novel approach to learning who to communicate to in a multi-agent setup, which is both interesting from a research perspective but also useful in practical applications of multi-agent communication. Moreover, this paper is well executed, with clear statements supported by sufficient experiments and baselines. Finally, R1 and R2 have expressed concerns regarding the low performance of IC3Net and TarMAC. Authors have provided an explanation in the author response with some more experiments with regards to team vs individual rewards.

Communication lays the foundation for human cooperation. It is also crucial for multi-agent cooperation. However, existing work focuses on broadcast communication, which is not only impractical but also leads to information redundancy that could even impair the learning process. To tackle these difficulties, we propose Individually Inferred Communication (I2C), a simple yet effective model to enable agents to learn a prior for agent-agent communication. The prior knowledge is learned via causal inference and realized by a feed-forward neural network that maps the agent's local observation to a belief about who to communicate with.